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ABSTRACT: 


Traffic Sign Recognition is used to give an audio description to a image sign traffic 
signs and thus warn a driver, and command or prohibit certain actions. A fast real-time and robust 
automatic traffic sign detection and recognition can support and disburden the driver and significantly 
increase driving safety and comfort. Automatic recognition of traffic signs is also important for 
automated intelligent driving vehicle or driver assistance systems. This paper presents a study 
recognize traffic sign patterns using Neural Networks technique. The images are pre-processed with 
several image processing techniques, such as, threshold techniques, Gaussian filter, Canny edge 
detection, Contour and Fit Ellipse. Then, the Neural Networks stages are performed to recognize the 
traffic sign patterns. The system is trained and validated to find the best network architecture. The 
experimental results show the highly accurate classifications of traffic sign patterns with complex 


background images and the computational cost of the proposed method.mental results 
KEYworps: Automatic, Neural network, Traffic signs, Recognition. 


1. INTRODUCTION 


The data base has been divided into: train, validation and test set. The first one is used to train the 
MLP. The second one is used as validation set during the MLP training to improve generalization. 
The last one is used to evaluate the performance of the trained MLP. To select network architecture 
for both generalization and approximation ability of a feedforward neural network, the number of 
layer and the number of hidden neurons per layer should be identification. Basically, using a single 
hidden layer tends to interact with global problems, therefore, in this paper, the single layer perceptron 
is employed with the verification of the number of hidden nodes. To find the hidden neurons for an 
appropriate network architecture, the use of a cross- validation approaches to cite. The method is to 
divide data set into a training set called training and a test set test. Then subdivide learning into two 
subsets: one to train the network learning, and one to validate the network validation. Train different 
network architectures on learning and evaluate their performance on validation. The best network has 
been selected and finally retrains this network architecture on training. The test set is utilized to test 
for generation ability of network the above figure shows the training set . The training set consists of 
two main data which are signs without problem and the signs with some distortion problems. The 
distortions are the stain color, noise, pixilated distortion, blur, and color distortion. These two set are 
fed to train the network simultaneously. shows the validation data which are different from the 


training set 
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2. RELATED WORK 


This section for highlights some recent related research works. Autonomas car is a recent research 
topic. We found many researches on this field and the working progress is too high.In 2019, Wei-Jong 
Yang et al. proposed an approach to recognize traffic sign. They worked with shaped based detection 
algorithms and for classification purpose they choose convolutional neural network. After simulation 
they got 97% sign recognition accuracy. In this paper author proposed SVM based classification 
algorithm to recognize traffic sign. Here, they considered 8 types of road signs. For training purpose 
they used 600 different images for each signs and for test purpose 120 images was considered. In this 
paper, they tested individual signs with real data and their accuracy level was 66.6% to 100%. in 2017 
proposed a Traffic Sign Recognition (TSR) system. Here, they were 

used HSV color model and deep CNN for automatic features extraction as a classifier. After this study, 
they achieved 97% accuracy. used Convolutional Neural Networks for recognizing traffic sign. Here, 
they found 99.18% accuracy on using Belgium Data and German Traffic Sign Benchmark (GTSDB). 
Where as they got 99.50% accuracy almost 0.32% improvement at accuracy. In 2016 at paper , Di 
Zang et al. classified their dataset using Support Vector Machine then detection part is done by CNN. 
Their accuracy was almost 96.50%, and they used GTSDB dataset. In 2017 at paper ,Ardianto et al. 
used SVM to classified object and got 91.5% accuracy in detection purpose and they also used 
GTSDB dataset. They improved it to add a feature called Histogram of Oriented Gradients (HOG) 
that help to increase its accuracy up to 98%. In 2017 Shi et al., applied SVM to detect which region 
of image contain a traffic sign. 


The GTSDB holds complete 900 images, where 800 for training and 100 for propose a Single Shot 
Detector algorithm combine with multi feature fusion and they called it MF-SSD. However, in this 
time they divided total 900 images into 600 training and 300 as a test data to detect traffic sign . Their 
main goal was to find out traffic sign detection methods to locate the regions-of-interest that contain 
traffic sign. They divided the methods into 3 steps- focused color, shape, and finally considered 
learning based methods . In 2016, for detecting and classifying the traffic signs they proposed an 
approach. It has 2 main steps: road sign detection, after that classification with recognition. To classify 
the traffic sign they used neural network and to complete this work they picked four types of traffic 
signs: Stop, No Entry, Give Way, and Speed Limit Sign. Considered total 3 hundreds sets images, and 
they got 90% and 88% accuracy for detection and recognition purpose . So, detection and recognition 
of traffic sign is necessary to build an autonomas car driving system. 


3. PROPOSED SOLUTION 


The purpose of this thesis is to explore different methods for helping 

computers interpret the real world visually, investigate solutions to those methods offered by the 
open-sourced computer vision library, OpenCV, and implement some of these in a Raspberry Pi base 
application for detecting and keeping track of objects. The main focus rests on the practical side of 
the project. The result of this thesis is a GNU/Linux based C/C++ application that is able to detect 
and keep track of objects by reading the pixel values of frames captured by the Raspberry Pi camera 
module. The application also transmits some useful information, such as coordinates and size, to other 
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computers on the network that send an appropriate query. The source code of the program is 
documented and can be developed further. 


Resize and Gray Scale 


Example image of traffic sign processing 


4. METHODOLOGY-convolutional neural network 


Convolutional neural network is a part of deep learning approch. Convolutiona neural network 
represents a vast breakthrough in image recognition. Many research fields focused on CNN to 
achieve the highest accurate result. ACNN model constracts with following four layers: 

e Convolutional layers 

e Relu function 

e Polling layers 

e Fully connected layer 

Convolution is the first step to create complete network. Here, Convolutioal 

operates on two images in 2D format. One as a input image, and another output image. It helps to 
understand the features from images by making relation between pixels. A convolution of two 
funcation f and g is defined with following the equation 


m 
F*DO=EIG) * f- j +2) 
jH 


Next, relu is placed after convolution operation. This “relu” fuction considered for executing non- 
linear operation. It performs on pixel wise operation and sets O insted of all negative values. There 
are other non-linear functions such as tanh or sigmoid also used in CNN but “relu” performs best 
amon them in different situations. On the other hand, Polling progressively decrese the size of the 
input images. Polling helps to decrese the number of dimentions and the number of claculation 
required. Polling also helps to control overfitting. There are many types of polling. But in this paper 
we used max polling. 

Fully connected layer, combines our features into output stage. It predicts the class level with better 
accuracy. The error is computed and then occured back propagation. This process is continue layer 
by layer. A sample flowchart for our proposed model on CNN and SVM is given in figure 
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Training DataSe 


Create Model 
(SVM , CNN) 


Sign Detection and 


Recognition 


Fig. 4.1. Flowchart of our CNN model 
5. DATA COLLECTION AND PREPARATION 


To complete this study a dataset was built from cropping video frame. We also collect some random 
video and crop traffic sign area to build a real dataset. Then, we categorized its own classes and split 
the whole data into training and validation dataset. We have total 1200 images to propose SVM and 
CNN model.We have considered 12 different classes and each class completed with 100images. Split 
the whole dataset as training and validation purpose. To generate a model with SVM, 80% data for 
trining and 20% data for testing purpose is considered. Meanwhile, for CNN classification we have 
divided dataset into two classes train and validation. For training purpose data volume was 1080 
images and 120 images for testing purpose, where total 12 different traffic signs was contained. Table 
1 to represnt the selected 12 traffic signs. Here, we illstrated evey traffic sign with sample an image.In 
the next section, we are attending to elaborate the work procedure of our proposed approach to detect 
and recognize traffic sign. 
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Description Traffic Sign Description Traffic Sign 


Turn Left Danger 


Stop 40km/h 


Only Left 30km/h 


Only Right Turn Right 


Road Merges Pedestrian 


Ahead 


Speed Breaker Bike 


a> Uk LD 
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Sign description table 
5. CONCLUSION 


This study discussed Traffic Sign Recognition (TSR) using Neural Networks technique. The images 
were pre-processed in stages with image processing techniques, such as, threshold technique, 
Gaussian filter, Canny edge detection, Contour and Fit Ellipse. Then, the Neural Networks stages 
were performed to recognize the traffic sign patterns. The main reason to select this method is to 
reduce the computational cost in to facilitate the real time implementation. The first strategy is to 
reduce the number of MLP inputs by pre-processing the traffic sign image, and the second strategy is 
to search for the best network architecture which reduced complexity by selecting a suitable error 
criterion for training. The system were trained with training data set, and validated with validating 
data set to find the best network architecture. The cross-validation technique was implemented with 
training data set, validating data set, and test set. The experiments show consistency results with 
accurate classifications of traffic sign pattems with complex background images. The processing time 
in each frame of image is provided which is satisfied to apply in the real applicationall of the above 
described algorithms OpenCV {RGB, HSV (color segmentation), Region of Interest, Speeded-Up 
Robust Features}, Convolutional Neural Networks, Normal Bayes Classifier contribute to the 
research done in development of this software: a highly accurate with fast response and robust Road 
Signs Detection and Recognition. The application was designed in android studio with the help of 
OpenCV library programmed by Java language. We have used the Region of Interest (ROI) with 
Speed Up Robust Feature (SURF) for object recognition. And Red Green Blue model, Hue Saturation 
Value 44model for color segmentation during day light and dark respectively. The results achieved 
from the experimentation cited above prove that our system is a real-time video/image processing 
software when exposed to a video sequence rather than to a standard approach of traffic sign detection 
in static images. The results of the test run also prove the convenience of the system. Examining the 
publicly available data set in real time shows that this system has fast processing speed and robust 
traffic sign recognition. This application has numerous applications like it will come in great aid to 
drivers with dis- abilities. In addition to this, it can help the drivers detect in advance the signs in 
critical weather conditions like fog, rain, thunderstorm, snowfall and so on. This application’s 
programming interface can be integrated with Maps and Navigation software which in turn can give 
seamless detection of the traffic signs with speech notification during navigation. Not only this, it can 
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tell at what speed the driver is going on currently and at what speed he/she needs to turn and that too 
in which direction. This is a very smooth application with precise and accurate predictions because 
of its design (coded in Java making the application faster compared to applications coded in Python) 
and the usage of OpenCV with neural networks 


6.RESULT & VALIDATION 
These are the outputs which are observed for our project while under working. Fig 5.1 sign 


recognition by sign board detectionn this Deep Learning project, we will build a model for the 
classification of traffic signs available in the image into many categories using a convolutional neural 


network(CNN) 
Phase #1: Detect ———_> 
traffic sign 
Phase #2: Recognize 
traffic sign (50km/h) 
Fig 5.1 sign recognition by sign board detection 
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